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State Transition Matrix and Diagram 


11.2.2 State Transition Matrix and Diagram 

We often list the transition probabilities in a matrix. The matrix is called the state transition matrix or transition probability 
matrix and is usually shown by P. Assuming the states are 1, 2, • • •, T, then the state transition matrix is given by 

Pll Pl2 ■ ■ ■ Plr 

P21 P22 • • • P2r 

P= ’ . 

-Prl Pr2 • • • Prr. 

Note that pij > 0, and for all i, we have 

r r 

^ j Pik ^ ^ P(.X-m +1 *) 

fc=1 k =1 

= 1 . 

This is because, given that we are in state i, the next state must be one of the possible states. Thus, when we sum over all 
the possible values of k, we should get one. That is, the rows of any state transition matrix must sum to one. 


State Transition Diagram: 

A Markov chain is usually shown by a state transition diagram. Consider a Markov chain with three possible states 1, 2, 
and 3 and the following transition probabilities 



Figure 11.7 shows the state transition diagram for the above Markov chain. In this diagram, there are three possible states 1, 
2, and 3, and the arrows from each state to other states show the transition probabilities Pij. When there is no arrow from 
state i to state j, it means that pij — 0. 
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Figure 11.7 - A state transition diagram. 


Example 11.4 

Consider the Markov chain shown in Figure 11.7. 

a. Find P{X^ — 3(^3 = 2). 

b. Find P(Xs — 1 |-X~2 — 1)- 

c. If we know P^Xq — 1) = find P^Xq — 1,X\ — 2). 

d. If we know P(Xq — 1) = y, find P(Xq — 1,X\ — 2,X2 = 3). 

• Solution 

° a. By definition 

P(X 4 = 3|X 3 = 2) = P23 = —. 

b. By definition 

P(x 3 = 1\X 2 = 1) = pu = ^. 

c. We can write 

P(X 0 = l,X x = 2)= P(X 0 = 1 )P(X 1 = 2\Xq = 1) 

1 

= 3‘ P12 

1 1 _ 1 

~ 3 ' 2 _ 6' 

d. We can write 

P(X 0 = 1,X 1 = 2,X 2 = 3) 

= P{X o = 1 )P{X 1 = 2|X 0 = 1 )P(X 2 = 3|X! = 2,X 0 = 1) 

= P(X o = l)P(Xi — 2 \Xq — V)P{X .2 — 3|Xi = 2) (by Markov property) 
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